Display device and method for controlling same

ABSTRACT

A display device includes: a display having transparency; a first detector which, in operation, detects a position indicated by an indicator on the display; a second detector which, in operation, detects position information and an orientation of the display; and a controller which, in operation, displays a virtual image corresponding to the position information and the orientation detected by the second detector on the display and changes a display mode of the virtual image according to the position indicated by the indicator and detected by the first detector.

BACKGROUND Technical Field

The present disclosure relates to a technique for providing augmentedreality by displaying a virtual image on a display.

Background Art

Augmented reality (AR) has been developed to enhance the real world byadding information to the real world. For example, there is a techniquefor displaying a virtual image on a transparent display (see, forexample, U.S. Pat. No. 8,941,683 and Japanese Patent No. 5649842).

However, the conventional technique faces a technical difficulty inchanging a display mode of a virtual image.

BRIEF SUMMARY

The present disclosure has been made in view of the technical issuedescribed above. It is an object of the present disclosure to provide atechnique for effectively changing a display mode of a virtual image.

According to one embodiment of the present disclosure, a display deviceincludes: a display having transparency; a first detector which, inoperation, detects a position indicated by an indicator on the display;a second detector which, in operation, detects position information andan orientation of the display; and a controller which, in operation,displays a virtual image corresponding to the position information andthe orientation on the display and changes a display mode of the virtualimage according to the position indicated by the indicator.

According to another embodiment of the present disclosure, a method forcontrolling a display device including a display having transparencyincludes: detecting a position indicated by an indicator on the display;detecting position information and an orientation of the display;displaying a virtual image corresponding to the position information andthe orientation on the display; and controlling changing of a displaymode of the virtual image according to the position indicated by theindicator.

According to the present disclosure, a display mode of a virtual imageis changed effectively.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are views of the external appearance of a display deviceaccording to a first embodiment of the present disclosure;

FIG. 2 is a schematic block diagram depicting functional blocks of thedisplay device depicted in FIGS. 1A and 1B;

FIG. 3A is a diagram depicting a display example in an AR1 mode; FIG. 3Bis a diagram depicting a display example in a non-AR mode;

FIGS. 4A to 4C are diagrams depicting an example of switching betweenthe AR1 mode, an AR2 mode, and the non-AR mode;

FIG. 5 is a flowchart of the entire operation of a controller depictedin FIG. 2;

FIG. 6 is a flowchart of the operation of the controller, which isdepicted in FIG. 2, in the non-AR mode;

FIG. 7 is a flowchart of the operation of the controller, which isdepicted in FIG. 2, in the AR1 mode;

FIG. 8 is a flowchart of the operation of the controller, which isdepicted in FIG. 2, in the AR2 mode;

FIG. 9 is a diagram depicting the display device and an equatorial mountequipped with a camera;

FIG. 10 is a diagram depicting how the display device according to asecond embodiment of the present disclosure is used;

FIG. 11 is a diagram depicting how the display device according to athird embodiment of the present disclosure is used;

FIGS. 12A and 12B are diagrams depicting how the display deviceaccording to a fourth embodiment of the present disclosure is used;

FIGS. 13A and 13B are diagrams depicting how the display deviceaccording to a modification of the fourth embodiment of the presentdisclosure is used;

FIG. 14 is a diagram depicting how the display device according to afifth embodiment of the present disclosure is used;

FIGS. 15A and 15B are diagrams depicting how the display deviceaccording to a sixth embodiment of the present disclosure is used;

FIG. 16 is a flowchart of the operation of the controller of the displaydevice according to the sixth embodiment of the present disclosure;

FIGS. 17A and 17B are diagrams depicting how the display deviceaccording to a modification of the sixth embodiment of the presentdisclosure is used;

FIGS. 18A to 18E are diagrams depicting how the display device accordingto a seventh embodiment of the present disclosure is used;

FIG. 19 is a flowchart of the operation of the controller of the displaydevice according to the seventh embodiment of the present disclosure;and

FIGS. 20A and 20B are diagrams depicting how the display deviceaccording to an eighth embodiment of the present disclosure is used.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, a display device according to embodiments of the presentdisclosure will be described with reference to the drawings.

FIGS. 1A and 1B are views of the external appearance of a display device1 according to a first embodiment of the present disclosure.Specifically, FIG. 1A depicts the display device 1 viewed from the frontside of the display device 1. FIG. 1B depicts the display device 1viewed from the back side of the display device 1. As depicted in FIGS.1A and 1B, the display device 1 has a circular shape and includes adisplay 2 at the center of the display device 1. The display 2 has acircular shape. A housing 3 has a donut shape and is provided around thedisplay 2. The housing 3 includes an imager 4 (built-in camera) on theback surface of the housing 3. As described in detail later, an externalcamera (including a camera coupled to a telephoto lens, a telescope, orthe like) provided separately from the display device 1 may be used asthe imager 4. In this case, communication may be performed between thedisplay device 1 and the external camera. When the user uses the displaydevice 1, the user views the display 2 by holding the housing 3 by ahand.

FIG. 2 is a schematic block diagram depicting functional blocks of thedisplay device 1. As depicted in the FIG. 2, the display device 1functionally includes a first detector 5, a second detector 6, acommunication device 7, a storage 8, and a controller 9, in addition tothe display 2 and the imager 4 depicted in FIGS. 1A and 1B. The displaydevice 1 can be used together with an electronic pen 10. The electronicpen 10 acts as an indicator.

The display 2 is a transparent liquid crystal display (LCD) panel. Thetransmittance of the display 2 can be changed according to the value ofthe voltage applied from the controller 9. To change the transmittance,for example, a liquid crystal film (electronic shutter) is attached onthe back surface of the transparent LCD panel so that the orientationdirection is changed according to the value of the voltage applied fromthe controller 9. When the transmittance of the display 2 is greaterthan 0%, the user views, through the display 2, an object in the realworld that exists on the other side of the display device 1. Therefore,when a virtual image is displayed on the display 2, the virtual imagedisplayed on the display 2 is superimposed on the object in the realworld, allowing the user to view the object in the real world and thevirtual image superimposed on each other on the display 2. In thismanner, augmented reality (AR) (an AR1 mode described later) isrealized.

The imager 4 is a camera built in the display device 1 and is capable ofoutputting a captured video to the controller 9 in real time. Thecontroller 9 outputs the video input from the imager 4 to the display 2in real time. When the video output from the imager 4 is being displayedon the display 2, the user views the video of an object in the realworld captured by the imager 4. Therefore, when a virtual image issuperimposed on the video output from the imager 4 and displayed, theuser views the object in the real world and the virtual image displayedon the display 2 that are superimposed on each other. In this manner aswell, augmented reality (AR) (an AR2 mode described later) is realized.

The first detector 5 is a transparent sensor device or circuit arrangedon the back surface of the display 2 and functions to detect a positionindicated by the indicator on the display 2. While various methods suchas an active electrostatic coupling method, a capacitance method, anelectromagnetic induction method, and a pressure sensing method areavailable as a method of detecting a position indicated by the firstdetector 5, the active electrostatic coupling method is preferable forthe display device 1. This is because the active electrostatic couplingmethod supports detection of both a touch by the electronic pen 10 and atouch by a finger. The following description will be given on thepremise that the display device 1 employs the active electrostaticcoupling method.

For example, the first detector 5 includes a plurality of first linearconductors (not depicted) and a plurality of second linear conductors(not depicted). Each of the plurality of first linear conductors extendsin the X direction. Each of the plurality of second linear conductorsextends in the Y direction perpendicular to the X direction on the panelplane. The first detector 5 detects a position indicated by theelectronic pen 10 or the finger using the plurality of first and secondlinear conductors.

First, description will be given with regard to a case where the firstdetector 5 detects a position indicated by the electronic pen 10. Theelectronic pen 10 employing the active electrostatic coupling method hasa function of transmitting position detection signals to the displaydevice 1. The first detector 5 detects a position detection signal ineach of the plurality of first and second linear conductors anddetermines the strength of the position detection signal received byeach of the first and second linear conductors. Greater strength of thereceived position detection signal indicates that the distance betweenthe linear conductor and the electronic pen 10 is small. From thestrength of the received position detection signals, therefore, thefirst detector 5 can detect a position indicated by the electronic pen10. The first detector 5 outputs the detected position to the controller9 as coordinate data including two-dimensional X and Y coordinatevalues.

Next, a description will be given with regard to a case where the firstdetector 5 detects a position indicated by the finger. The firstdetector 5 sequentially supplies a signal to each of the plurality offirst linear conductors and sequentially causes each of the plurality ofsecond linear conductors to receive the signal. The first detector 5determines the strength of the received signal for each of combinationsof the first and second linear conductors. Specifically, when the fingerapproaches the intersection of one of the combinations of the first andsecond linear conductors, part of the current flowing from the firstlinear conductor to the second linear conductor is absorbed by the humanbody. This, as a result, reduces the strength of the signal received bythe second linear conductor. By determining the strength of the receivedsignals as described above, therefore, the first detector 5 candetermine the intersection at which the finger is approaching, that is,the position indicated by the finger. As in the case of the electronicpen 10, the first detector 5 outputs the detected position to thecontroller 9 as coordinate data including two-dimensional X and Ycoordinate values.

The electronic pen 10 is capable of transmitting various types ofinformation such as writing pressure information, side switchinformation, and a unique identifier (ID), in addition to the positiondetection signals. The writing pressure information indicates pressure(writing pressure value) applied to the pen tip. The side switchinformation indicates on or off of a side switch provided on the sidesurface of the electronic pen 10. The unique ID is stored in theelectronic pen 10 in advance. When the first detector 5 receives thesepieces of information from the electronic pen 10, the first detector 5combines these pieces of information with the coordinate data indicatingthe position indicated by the electronic pen 10 and transmits thecombined information to the controller 9 as one piece of information.

The second detector 6 is a functional device or circuit that detectsposition information (for example, the latitude and longitude) andorientation of the display device 1. Among the functions of the seconddetector 6, the function of detecting the position information isimplemented using a GPS receiver, for example. The function of detectingthe orientation is implemented using, for example, a sensor capable ofdetecting the acceleration, magnetism, angular velocity, and the like.The second detector 6 transmits the detected position information andorientation to the controller 9.

The communication device 7 is a communication device that performs datacommunication with the electronic pen 10 and an arbitrary informationterminal (not depicted). It is noted that the communication performed bythe communication device 7 is different from the communication performedbetween the electronic pen 10 and the first detector 5. Thecommunication device 7 may communicate with the electronic pen 10 or thearbitrary information terminal by wireless communication or maycommunicate with the electronic pen 10 or the arbitrary informationterminal by wired communication. In one example, communication betweenthe communication device 7 and the electronic pen 10 is performed usingBluetooth (registered trademark), while communication between thecommunication device 7 and the information terminal is performed by awireless local area network (LAN).

The storage 8 is a functional device that stores a computer program anddata. The storage 8 is, for example, implemented by a storage devicesuch as a dynamic random access memory (DRAM), a flash memory, or a harddisk or implemented by a combination of these storage devices.

The controller 9 is a processor that executes each operation of thedisplay device 1 described in the present embodiment by reading andexecuting the program stored in the storage 8. The controller 9 iscoupled to each component depicted in FIG. 2 through a bus (notdepicted) or the like.

Operations performed by the controller 9 include an operation of causingthe display 2 to display a virtual image corresponding to the positioninformation and orientation detected by the second detector 6 andchanging a display mode of the virtual image according to the indicatedposition detected by the first detector 5. Specifically, the controller9 performs this operation by switching its mode to one of three modes,i. e., in the AR1 mode, the AR2 mode, and a non-AR mode.

The AR1 mode realizes augmented reality by displaying a virtual imagesuperimposed on an object in the real world viewed through thetransparent display 2. The controller 9 in this mode sets thetransmittance of the display 2 to a first value. The first value isgreater than 0%. For example, the first value is the maximum value(preferably 100%) of the transmittance. Subsequently, the controller 9forms a virtual image corresponding to the position information andorientation detected by the second detector 6, and outputs the virtualimage to the display 2.

FIG. 3A is a diagram depicting a display example in the AR1 mode. In theexample in FIG. 3A, a real object Mt. Fuji R1 exists on the other sideof the display 2 and the display device 1 is held by the user. In thisexample, while the controller 9 is in the AR1 mode, the display 2 istransparent except the area in which a human image V1, which is avirtual image, is displayed. Therefore, the user is able to view,through the display 2, the Mt. Fuji R1 that exists on the other side ofthe display device 1. As a result, the human image V1 appears to besuperimposed on the Mt. Fuji R1, allowing the user to experience theaugmented reality.

The AR2 mode realizes augmented reality by displaying a virtual imagesuperimposed on a video output from the imager 4. The controller 9 inthis mode sets the transmittance of the display 2 to a third valuesmaller than the first value and causes the imager 4 to start capturinga video. The third value is, for example, the minimum value (preferably0%) of the transmittance. Then, the controller 9 sequentially outputsthe video supplied in real time from the imager 4 to the display 2,while forming a virtual image corresponding to the position informationand orientation detected by the second detector 6 and outputting thevirtual image to the display 2. As a result, a real object displayedthrough the imager 4 and the virtual image are superimposed on eachother and displayed on the display 2. The real object and the virtualimage are displayed similarly to the AR1 mode depicted in FIG. 3A,allowing the user to experience the augmented reality.

The non-AR mode causes the display 2 to display a virtual image only.The controller 9 in this mode sets the transmittance of the display 2 toa second value smaller than the first value. For example, the secondvalue is the minimum value (preferably 0%) of the transmittance.Subsequently, the controller 9 forms a virtual image corresponding tothe position information and orientation detected by the second detector6 and outputs the virtual image to the display 2.

FIG. 3B is a diagram depicting a display example in the non-AR mode. Inthe example in FIG. 3B, the controller 9 has been switched to the non-ARmode during the state in FIG. 3A. Entering the non-AR mode makes thedisplay 2 opaque, disabling the user to view, at least on the display 2,the Mt. Fuji R1 that exists on the other side of the display 2. As aresult, the user views the human image V1, which is a virtual image.

Additionally, the operations performed by the controller 9 also includean operation of generating stroke data on the basis of a series ofindicated positions detected by the first detector 5 and storing thestroke data in the storage 8 while displaying the stroke data on thedisplay 2.

How the controller 9 changes the display mode of the virtual image willbe specifically described below, using an example in which the displaydevice 1 is used for astronomical observation.

FIGS. 4A to 4C are diagrams depicting an example of switching betweenthe AR1 mode, the AR2 mode, and the non-AR mode. Specifically, FIG. 4Adepicts a state (non-AR mode) in which a planisphere is displayed on thedisplay 2. FIG. 4B depicts a state (AR1 mode) in which astronomicalobjects in the real starry sky are visible through the display 2. FIG.4C depicts a state (AR2 mode) in which an image of the astronomicalobjects captured by the imager 4 is displayed on the display 2.

By referring to FIG. 4A, the controller 9 has a function of displaying,on the display 2, the planisphere (virtual image) widely used forastronomical observation. When the planisphere is displayed, thecontroller 9 is in the non-AR mode and displays the virtual image on thedisplay 2.

Here, the structure of a planisphere will be briefly described. Theplanisphere includes a mask arranged on a star chart. The mask has anelliptical opening to display astronomical objects above the horizon inthe real starry sky.

The planisphere displayed on the display 2 in the example in FIG. 4A hasbasically a similar structure to the structure of the planispheredescribed above and includes an astronomical object displaying area VAand a mask area VM. The astronomical object displaying area VAcorresponds to the opening of the mask. The mask area VM corresponds tothe mask (area other than the opening).

The controller 9 enters the non-AR mode to display the planisphere.Then, the controller 9 obtains the current date and time from a built-incalendar and a built-in clock, not depicted, while obtaining the currentlatitude and longitude from the position information detected by thesecond detector 6. The controller 9 displays, in the astronomical objectdisplaying area VA, an all-sky image corresponding to the obtainedcurrent date and time and the obtained current latitude and longitude,together with additional information such as right ascension anddeclination. Accordingly, the user can view the entire starry sky in thecurrent position and the current date and time in the astronomicalobject displaying area VA. The mask area VM includes a latitude andlongitude displaying section V4 and a date and time displaying sectionV5. The controller 9 displays the obtained current date and time in thedate and time displaying section V5 while displaying the obtainedcurrent latitude and longitude in the latitude and longitude displayingsection V4.

The controller 9 displays a mark V6 in the astronomical objectdisplaying area VA. The mark V6 indicates the orientation detected bythe second detector 6. The mark V6 is a double-circle image, forexample. The controller 9 adjusts the display position of the mark V6such that the center of the double circle matches the orientationdetected by the second detector 6. Displaying the mark V6 allows theuser to broadly understand which part of the starry sky the user isoriented to.

The user can easily find a constellation in the real starry sky by usingthe planisphere depicted in FIG. 4A. For example, when the user wants toobserve the Orion constellation, the user changes the orientation of thedisplay device 1 such that the mark V6 approaches an Orion constellationimage V7 in the astronomical object displaying area VA (this movement isindicated by an arrow A). The display device 1 turned in this manner isoriented in the direction of the Orion constellation in the real starrysky. Thus, the real Orion constellation appears in front of the userwhen the user puts the display device 1 out of view.

When the user touches an arbitrary position in the astronomical objectdisplaying area VA while the planisphere is displayed on the display 2,the controller 9 changes the display state of the display 2 by changingits mode in response to this touch. The controller 9 switches its modeto the AR1 mode or the AR2 mode depending on the touch operationperformed by the user. Description will be given for each of the AR1mode and the AR2 mode.

When the touch operation is a so-called single-tap operation, forexample, the controller 9 enters the AR1 mode and sets the transmittanceof the display 2 to the first value by controlling the voltage appliedto the display 2. In addition, the controller 9 deletes the planispherefrom the display 2 and instead displays, on the display 2, informationon stars that have been displayed in the vicinity of the mark V6 in theastronomical object displaying area VA. The information on starsincludes, for example, the shape of a constellation and the name of anastronomical object and is displayed on the display 2 as a virtualimage. Accordingly, the user can check, for example, the shape of theconstellation in the real starry sky.

FIG. 4B depicts a case where the mark V6 has been located in thevicinity of the Orion constellation image V7 in the astronomical objectdisplaying area VA when the user touches an arbitrary position in theastronomical object displaying area VA depicted in FIG. 4A. That is,FIG. 4B depicts a case where the display device 1 has been oriented inthe direction of the Orion constellation. In this case, the user viewsthe real Orion constellation through the display 2 that has been turnedtransparent. As depicted in FIG. 4B, the controller 9 also displays aconstellation image V8 on the display 2. The constellation image V8represents a constellation within a predetermined range centered on theOrion constellation. As a result, the constellation image V8superimposed on the real starry sky is displayed on the display 2. Inaddition to the constellation image V8 depicted in FIG. 4B, virtualimages displayed in this manner may also include the names (includingMessier and new general catalogue (NGC)) of stars, star clusters,nebulae, planets, and comets or the like, and artificial satellites theequator (including the right ascension and declination), and theecliptic or the like.

When the touch operation is a so-called double-tap operation, forexample, the controller 9 enters the AR2 mode and sets the transmittanceof the display 2 to the third value by controlling the voltage appliedto the display 2. In addition, the controller 9 deletes the planispherefrom the display 2 while activating the imager 4. The controller 9sequentially outputs a video supplied in real time from the imager 4 tothe display 2, and the video output from the imager 4 is displayed onthe display 2. At the same time, virtual images similar to the virtualimages displayed in the AR1 mode may also be displayed on the display 2.As in the case of the AR1 mode, the user can check, for example, theshape of a constellation in the real starry sky.

FIG. 4C depicts a case where the user has touched the Great Orion Nebula(a nebula in the vicinity of the center of the Orion constellation imageV7) in the astronomical object displaying area VA. The controller 9displays a video output from the imager 4 on the display 2. This allowsthe user to view, on the display 2, even an astronomical object that isotherwise difficult to be viewed with the naked eyes (that is, difficultto be viewed in the display state in FIG. 4B).

In both of the display depicted in FIG. 4B and the display depicted inFIG. 4C, the controller 9 displays, as one of the virtual images, areturn button V9 to return to the planisphere. When the user touches thereturn button V9, the controller 9 returns its mode to the non-AR modeand displays the planisphere depicted in FIG. 4A on the display 2 again.Accordingly, the user can check the planisphere again.

When the user touches an area other than the return button V9 during thedisplay depicted in FIG. 4B, the controller 9 sets its mode to the AR2mode to switch to the display depicted in FIG. 4C. In this case, thearea displayed on the entire display 2 preferably corresponds to theposition touched by the user so that the user can view the video ofastronomical objects at the touched position. When the user touches anarea other than the return button V9 during the display depicted in FIG.4C, the controller 9 sets its mode to the AR1 mode to switch to thedisplay depicted in FIG. 4B. Accordingly, the user can view astronomicalobjects in a wider range.

The controller 9 may change the display state of the display 2 by atrigger other than a touch. For example, when the mark V6 has comecloser to a predetermined constellation (for example, the Orionconstellation image V7) during the display of the planisphere, thedisplay state may be switched to the display depicted in FIG. 4B.Further, when the orientation of the display 2 has been away from theconstellation (for example, the Orion constellation image V7) beingdisplayed during the display depicted in FIG. 4B, the display state maybe switched to the display of the planisphere.

The operation of the controller 9 will be described in more detail withreference to a flowchart of the controller 9. As an example, the displaydevice 1 is used for astronomical observation (see FIGS. 4A to 4C).

FIGS. 5 to 8 are flowcharts of the operation of the controller 9. Asdepicted in FIG. 5, the controller 9 starts operation in the non-AR mode(S1).

FIG. 6 is a flowchart of the operation of the controller 9 in the non-ARmode. As depicted in the FIG. 6, the controller 9 in the non-AR modesets the transmittance of the display 2 to the second value (S10). InFIG. 6, the operation at S10 is circled with a broken line. This isbecause the operation at S10 is not performed when the transmittance ofthe display 2 has already been set to the second value.

Next, the controller 9 displays the planisphere depicted in FIG. 4A onthe display 2 (S11). Subsequently, the controller 9 obtains the currentdate and time from the built-in clock, not depicted, and displays thecurrent date and time in the date and time displaying section V5depicted in FIG. 4A (S12). The controller 9 also obtains positioninformation of the display device 1 from the second detector 6 anddisplays the position information in the latitude and longitudedisplaying section V4 depicted in FIG. 4A (S13). Further, the controller9 obtains an all-sky image (virtual image) corresponding to the obtainedcurrent time and position information and displays the all-sky image inthe astronomical object displaying area VA depicted in FIG. 4A (S14).The controller 9 also obtains the orientation of the display device 1from the second detector 6 and displays the mark V6 depicted in FIG. 4Ain a position corresponding to the obtained orientation in theastronomical object displaying area VA (S15).

Referring back to FIG. 5, the controller 9 that has performed theoperation at S1 determines whether the user has touched the astronomicalobject displaying area VA by referring to the data output from the firstdetector 5 (S2). When the controller 9 determines that the user has nottouched the astronomical object displaying area VA, the controller 9repeats the operation at S1. When the controller 9 determines that theuser has touched the astronomical object displaying area VA, thecontroller 9 determines what touch operation has been performed byfurther referring to the data output from the first detector 5 (S3).When the controller 9 determines that the touch operation has been afirst operation (for example, a single tap), the controller 9 starts theoperation in the AR1 mode (S4). When the controller 9 determines thatthe touch operation has been a second operation (for example, a doubletap), the controller 9 starts the operation in the AR2 mode (S7).

FIG. 7 is a flowchart of the operation of the controller 9 in the AR1mode (S4). As depicted in the FIG. 7, the controller 9 that has enteredthe AR1 mode sets the transmittance of the display 2 to the first value(S20). At this point, the planisphere displayed in the non-AR mode isdeleted. In FIG. 7, the operation at S20 is circled with a broken line.This is because the operation at S20 is not performed when thetransmittance of the display 2 has already been set to the first value.

Next, the controller 9 obtains the current date and time, positioninformation, and orientation, as in the case of the non-AR mode (S21).The controller 9 displays a virtual image corresponding to the obtainedcurrent date and time, position information, and orientation on thedisplay 2 (S22).

FIG. 8 is a flowchart of the operation of the controller 9 in the AR2mode (S7). As depicted in the FIG. 8, the controller 9 that has enteredthe AR2 mode sets the transmittance of the display 2 to the third value(S30). In FIG. 8, the operation at S30 is circled with a broken line.This is because the operation at S30 is not performed when thetransmittance of the display 2 has already been set to the third value.

Next, the controller 9 displays a video output from the imager 4 on thedisplay 2 on the basis of the position touched by the user (S31). Atthis point, the planisphere displayed in the non-AR mode is deleted.

Subsequently, the controller 9 obtains the current date and time,position information, and orientation, as in the case of the non-AR mode(S32). On the basis of the obtained current date and time, positioninformation, and orientation, the controller 9 estimates the positionrange of the real world being displayed on the display 2 (S33). Then,the controller 9 displays, on the display 2, a virtual imagecorresponding to the estimated position range (S34). Accordingly, thevirtual image corresponding to the astronomical objects actually beingdisplayed on the display 2 can be displayed.

Referring back to FIG. 5, the controller 9 that has performed theoperation at S4 determines whether the user has touched the display 2 byreferring to the data output from the first detector 5 (S5). When thecontroller 9 determines that the user has not touched the display 2, thecontroller 9 repeats the operation at S4. When the controller 9determines that the user has touched the display 2, the controller 9determines the touched area by further referring to the data output fromthe first detector 5 (S6). When the controller 9 determines that thetouched area is within a predetermined area (for example, the returnbutton V9 depicted in FIG. 4B), the controller 9 starts the operation inthe non-AR mode (S1). When the controller 9 determines that the touchedarea is the other area, the controller 9 starts the operation in the AR2mode (S7).

The controller 9 that has performed the operation at S7 determineswhether the user has touched the display 2 by referring to the dataoutput from the first detector 5 (S8). When the controller 9 determinesthat the user has not touched the display 2, the controller 9 repeatsthe operation at S7. When the controller 9 determines that the user hastouched the display 2, the controller 9 determines the touched area byfurther referring to the data output from the first detector 5 (S9).When the controller 9 determines that the touched area is within apredetermined area (for example, the return button V9 depicted in FIG.4C), the controller 9 starts the operation in the non-AR mode (S1). Whenthe controller 9 determines that the touched area is the other area, thecontroller 9 starts the operation in the AR1 mode (S4).

As described above, the display device 1 according to the presentembodiment is capable of effectively changing the display mode of thevirtual image representing the augmented reality on the display 2according to the touch operation performed by the user.

It is noted that the imager 4 may be attached to an equatorial mount andthe display device 1 may be used as a controller for this equatorialmount. The details will be described below with reference to FIGS. 9 and10.

FIG. 9 is a diagram depicting the display device 1 and an equatorialmount 21 equipped with a camera 20. The camera 20 acts as the imager 4.Each of the camera 20 and the equatorial mount 21 is capable ofcommunicating with the controller 9 by wireless communication throughthe communication device 7. A telephoto lens is preferably attached tothe camera 20 to allow the user to observe an astronomical object at amagnification as depicted in FIG. 4C. An image of an astronomical objectcaptured by the camera 20 may be displayed in real time on the displaydevice 1. Alternatively, an image captured and stored in advance may bedisplayed. Still alternatively, an image may be captured withinpredetermined exposure time and the captured image may be displayed.

As a preparation, the user aligns the polar axis of the equatorial mount21. Next, the controller 9 displays the planisphere depicted in FIG. 4Aon the display 2 of the display device 1. When the user touches anarbitrary position in the astronomical object displaying area VA, thecontroller 9 enables an automatic tracking function of the equatorialmount 21 to keep the camera 20 oriented in the direction of anastronomical object (hereinafter referred to as “tracking targetastronomical object”) corresponding to the position touched by the user.The controller 9 also sets its mode to the AR2 mode. In the AR2 mode, avideo captured by the camera 20 is displayed on the display 2. With thisconfiguration, even after time passes, the tracking target astronomicalobject continues to be displayed on the display 2. Accordingly, the usercan view, on the display 2, the tracking target astronomical object andthe virtual image superimposed on the tracking target astronomicalobject.

A second embodiment of the present disclosure will now be described. Inaddition to the functions described in the first embodiment, the displaydevice 1 according to the present embodiment has a function ofcooperating with another display device 1, allowing the user to view, onthe display 2, a video captured by another person's camera.

FIG. 10 is a diagram depicting how the display device 1 according to thepresent embodiment is used. Each of display devices 1 a and 1 b depictedin FIG. 10 represents the display device 1 according to the presentembodiment. The display device 1 a is used to observe the starry sky inBeijing while the display device 1 b is used to observe the starry skyin Tokyo. In the example in FIG. 10, the display devices 1 a and 1 b arein the AR2 mode and display videos received from external cameras 20 aand 20 b, respectively (see FIG. 4C). A mark Va indicates theorientation of the display device 1 a. A mark Vb indicates theorientation of the display device 1 b.

The display devices 1 a and 1 b are capable of communicating with eachother through a network 30. Specifically, each of the display devices 1a and 1 b performs communication using the communication device 7depicted in FIG. 2. Each display device 1 according to the presentembodiment is assigned an identification number in advance to identifyone another and stores the identification number in the storage 8 inadvance. When the controller 9 of the display device 1 b startscommunication with the display device 1 a, the controller 9 of thedisplay device 1 b transmits the identification number stored in thestorage 8. The controller 9 of the display device 1 a performsauthentication using this identification number to determine whethercommunication can be performed with the display device 1 b. When thecontroller 9 of the display device 1 a determines that communication canbe performed with the display device 1 b, the controller 9 of thedisplay device 1 a starts communication with the display device 1 b.

After communication starts with the display device 1 b, the controller 9of the display device 1 a displays, on the display 2 of the displaydevice 1 a, not only the mark Va indicating the orientation of thedisplay device 1 a but also the mark Vb indicating the orientation ofthe display device 1 b. The mark Va and the mark Vb are different fromeach other in appearances such as the shape and color. Similarly, thecontroller 9 of the display device 1 b displays, on the display 2 of thedisplay device 1 b, not only the mark Vb indicating the orientation ofthe display device 1 b but also the mark Va indicating the orientationof the display device 1 a. When the mark Vb in the astronomical objectdisplaying area VA is touched on the display device 1 a, the controller9 of the display device 1 b distributes (for example, streams), to thedisplay device 1 a, a video obtained from the external camera 20 b.Accordingly, the display device 1 a displays the video that is capturedby the camera 20 b and that can be viewed on the display device 1 b.Similarly, when the mark Va in the astronomical object displaying areaVA is touched on the display device 1 b, the controller 9 of the displaydevice 1 a distributes a video obtained from the external camera 20 a tothe display device 1 b. Accordingly, the display device 1 b displays thevideo that is captured by the camera 20 a and that can be viewed on thedisplay device 1 a.

In the present embodiment, it is sufficient if the display device 1 thatdistributes a video is capable of obtaining the video from the camera 20and distributing the video to other display devices 1. Therefore, thedisplay device 1 may be another type of device. For example, instead ofthe display device 1, a distribution server may be provided to stream avideo captured by the camera 20 to the display devices 1 around theworld. This example is particularly effective for a large telescope suchas the Hubble Space Telescope and astronomical telescopes installed atvarious astronomical observatories that have a large number of users whowant to view the video from the large telescope. In this example, thedistribution server distributes a video captured by a camera attached tothe large telescope.

A third embodiment of the present disclosure will now be described. Inaddition to the functions described in the first embodiment, the displaydevice 1 according to the present embodiment has a function ofmagnifying and displaying a virtual image on a screen or the like usinglight from an external light source (hereinafter referred to as“planetarium function”).

FIG. 11 is a diagram depicting how the display device 1 according to thepresent embodiment is used. When the controller 9 according to thepresent embodiment executes the planetarium function, the controller 9enters the AR1 mode and displays, on the display 2, a virtualastronomical object image V10 to be projected. The virtual astronomicalobject image V10 is an image depicting the starry sky. Morespecifically, the virtual astronomical object image V10 is a negativeimage having an opaque portion and a transparent portion. The opaqueportion is where an astronomical object exists. The portion other thanthe opaque portion is transparent.

When an external light source 31 irradiates one of the surfaces of thedisplay device 1 with light while the controller 9 displays the virtualastronomical object image V10 on the display 2 as described above, thevirtual astronomical object image V10 is projected on a screen 32located on the opposite side of the light source 31 with the displaydevice 1 interposed between the screen 32 and the light source 31.Accordingly, the user can enjoy the virtual astronomical object imageV10 magnified and displayed on the screen 32. The screen 32 ispreferably a wall of a room, for example.

A fourth embodiment of the present disclosure will now be described. Inaddition to the functions described in the first embodiment, the displaydevice 1 according to the present embodiment has a function of actinglike tracing paper.

FIGS. 12A and 12B are diagrams depicting how the display device 1according to the present embodiment is used. As depicted in FIG. 12A,when the display device 1 is used like tracing paper, the controller 9sets its mode to the AR1 mode and sets the transmittance of the display2 to the first value. The controller 9 also displays a switching buttonV11 (first image) on the display 2.

With the transmittance of the display 2 set to the first value, anobject in the real world on the other side of the display device 1 (Mt.Fuji in FIG. 12A) is displayed on the display 2. The user inputs a lineimage V12 using the electronic pen 10 to trace the object in the realworld. In this manner, the line image V12 is displayed on the display 2,and the user can trace the object in the real world.

When the user touches the switching button V11 (when the indicatedposition detected by the first detector 5 includes the position on theswitching button V11), the controller 9 changes its mode to the non-ARmode and sets the transmittance of the display 2 to the second value.Accordingly, the user can check the line image V12 on the display 2, asdepicted in FIG. 12B.

A slider may be used instead of the switching button V11. In the exampledepicted in FIGS. 13A and 13B, a slider V13 (second image) is displayedon the display 2, instead of the switching button V11. The controller 9controls the transmittance of the display 2 on the basis of theindicated position detected by the first detector 5 on the slider V13.Specifically, for example, when the indicated position is at one end ofthe slider V13, the controller 9 sets the transmittance of the display 2to the first value. When the indicated position is at the other end ofthe slider V13, the controller 9 sets the transmittance of the display 2to the second value or the third value. When the indicated position isin the middle of the slider V13, the transmittance of the display 2 isset to an intermediate value between the first value and the secondvalue or the third value. This example allows setting of theintermediate transmittance. By setting the transmittance toapproximately an intermediate value, therefore, the line image V12 canrelatively stand out, as depicted in FIG. 13B.

A fifth embodiment of the present disclosure will now be described. Inaddition to the functions described in the fourth embodiment, thedisplay device 1 according to the present embodiment has a function ofconfiguring a 3D image in the display device 1.

FIG. 14 is a diagram depicting how the display device 1 according to thepresent embodiment is used. In the example in FIG. 14, the user traces abuilding 33 from three directions D1 to D3 using the functions describedin the fourth embodiment, and stores a virtual image of the building 33generated for each direction in the storage 8. At this time, thecontroller 9 causes the storage 8 to store the virtual image generatedby the user in association with the orientation of the display device 1detected by the second detector 6 during tracing. After all the virtualimages have been stored, the controller 9 generates a 3D image bycombining the plurality of virtual images stored in the storage 8 on thebasis of the orientation of the display device 1 associated with eachvirtual image. Thus, the user can obtain the 3D image of the building 33by simply tracing the building 33 from a plurality of directions.

A sixth embodiment of the present disclosure will now be described. Inaddition to the functions described in the first embodiment, the displaydevice 1 according to the present embodiment has a game function. Thisgame is to find a monster in the real world. The monster is made up of avirtual image.

FIGS. 15A and 15B are diagrams depicting how the display device 1according to the present embodiment is used. In the initial statedepicted in FIG. 15A, the controller 9 according to the presentembodiment is in the AR1 mode, and the user views the landscape in thereal world on the other side of the display device 1 through the display2. In this state, the controller 9 according to the present embodimentdisplays a monster image V14 on the display 2. The monster image V14 isa virtual image. To display the monster image V14, the storage 8according to the present embodiment stores each of a plurality ofmonster images in association with corresponding position information inadvance.

FIG. 16 is a flowchart of the operation of the controller 9 of thedisplay device 1 according to the present embodiment. Hereinafter, theoperation of the controller 9 according to the present embodiment willbe described with reference to the flowchart.

The controller 9 enters the AR1 mode (S40) and sets the transmittance ofthe display 2 to the first value. The controller 9 estimates theposition range of the real world being displayed on the display 2 on thebasis of the position information and orientation detected by the seconddetector 6 (S41). When a monster image corresponding to a positionincluded in the estimated position range has been stored in the storage8, the controller 9 displays, on the display 2, the monster imagesuperimposed on the landscape in the real world, as depicted in FIG. 15A(S42).

At S42, the controller 9 controls an enlargement ratio on the basis ofthe distance between the position information, which is stored in thestorage 8 in association with each monster image, and the currentposition of the display device 1. Accordingly, the user experience issuch that the monster that is farther away appears smaller in size.Controlling the enlargement ratio, however, may lead to a situationwhere the user can recognize the monster in the distance but cannot viewthe monster well. The user in this situation often desires to enlargethe monster for better view. The operations in and after S43 areperformed to satisfy such user's desire.

The controller 9 determines whether the monster image being displayedhas been touched (S43). When the controller 9 determines that themonster image has not been touched, the controller 9 returns to S41 tocontinue the operation. When the controller 9 determines that themonster image has been touched, the controller 9 enters the AR2 mode(S44) and determines a cutout range (position and size) of a videooutput from the imager 4 on the basis of the position of the touchedmonster image (S45). The controller 9 determines the position of thecutout range so as to include the monster image in the cutout range.Preferably, the controller 9 makes the cutout range smaller in size whenthe distance from the monster (the distance between the positioninformation stored in the storage 8 in association with the monsterimage and the current position of the display device 1) is greater.Alternatively, the user may set the size of the cutout range in advance.

The controller 9 changes the enlargement ratio of the monster imageaccording to the size of the cutout range determined at S45 (S46). Forexample, when the size of the cutout range corresponds to a quarter ofthe area of the video output from the imager 4, the controller 9 setsthe enlargement ratio of the monster image such that the monster imageis enlarged in area by four times.

According to the cutout range determined at S45, the controller 9 cutsout the video output from the imager 4 and enlarges and displays thecutout video on the entire display 2, as depicted in FIG. 15B (S47). Thecontroller 9 estimates the position range of the real world beingdisplayed on the display 2 on the basis of the position information andorientation detected by the second detector 6 (S48). The controller 9determines whether the position range includes the positioncorresponding to the touched monster image (S49). When the controller 9determines that the position range does not include the positioncorresponding to the touched monster image, the controller 9 returns tothe operation in the AR1 mode at S40. When the controller 9 determinesthat the position range includes the position corresponding to thetouched monster image, the controller 9 displays the touched monsterimage with the enlargement ratio determined at S46, as depicted in FIG.15B (S50). The controller 9 returns to S47 to continue the operation inthe AR2 mode.

The determination at S49 is made because when the user moves the displaydevice 1, the monster may go out of the range of the real world beingdisplayed on the display 2. When the controller 9 determines at S49 thatthe position range does not include the position corresponding to thetouched monster image, the controller 9 returns to the AR1 mode. Withthis configuration, when the monster image that has been displayed is nolonger displayed on the display 2, the controller 9 can promptly returnto the AR1 mode.

According to the present embodiment, the user can enlarge and view themonster in the distance by touching the monster image. Moreover, sincethe video of the surrounding real world is enlarged at the same time,the user can view the enlarged monster image without a feeling ofstrangeness.

FIGS. 17A and 17B are diagrams depicting how the display device 1according to a modification of the present embodiment is used. Thedisplay device 1 according to the present modification is different fromthe display device 1 according to the present embodiment in that whendisplayed, the video output from the imager 4 is not increased in sizebut reduced in size.

In the initial state depicted in FIG. 17A, the controller 9 is in theAR1 mode as in FIG. 15A. The controller 9 of the display device 1according to the present modification displays a switching button V15 onthe display 2. When the user touches the switching button V15, thecontroller 9 switches its mode to the AR2 mode and sets themagnification of the imager 4 to a predetermined value smaller than 1.Then, the controller 9 switches the display on the display 2 to the(entire) video output from the imager 4, which is a camera. Thecontroller 9 also displays, on the display 2, the monster image with anenlargement ratio (reduction ratio) equal to the magnification of theimager 4. Accordingly, as depicted in FIG. 17B, a wider range of videois displayed on the display 2, together with the monster image.

Since the user does not know the position of the monster (the positioninformation stored as the position of the monster image in the storage8) beforehand, the user may not successfully be able to capture themonster image on the display 2, as depicted in FIG. 17A. Even in such acase, the user can view a wider range by pressing the switching buttonV15. Therefore, the user can search for the monster more easily.

A seventh embodiment of the present disclosure will now be described. Inaddition to the functions described in the first embodiment, the displaydevice 1 according to the present embodiment has a function ofdisplaying a medical image. Specifically, when the controller 9 is inthe AR1 mode, a fluoroscopic image such as an X-ray image is displayed.When the controller 9 is in the non-AR mode, a cross-sectional image ofan arbitrary cross-section is displayed. The cross-sectional image isgenerated from volume data such as computed tomography (CT) and magneticresonance imaging (MRI).

FIGS. 18A to 18E are diagrams depicting how the display device 1according to the present embodiment is used. The above-describedfluoroscopic image and volume data are associated with each other on thebasis of patient information (ID) and stored in the storage 8 accordingto the present embodiment.

FIG. 19 is a flowchart of the operation of the controller 9 of thedisplay device 1 according to the present embodiment. Hereinafter, theoperation of the controller 9 according to the present embodiment willbe described with reference to the flowchart.

In order to determine the initial position, the controller 9 instructsthe user to place the display device 1, for example, on the head of theuser with the display 2 of the display device 1 facing upward (forexample, this instruction information is displayed on the display 2).After the initial position has been set, various sensors of the seconddetector 6 detect movement from the initial position. The controller 9enters the AR1 mode (S60) and sets the transmittance of the display 2 tothe first value. As depicted in FIG. 18A, a fluoroscopic image V16 isdisplayed on the display 2 on the basis of the information detected bythe second detector 6 (S61). The fluoroscopic image V16 corresponds tothe position of the patient viewed from the front. At this time, thecontroller 9 also displays a switching button V17 on the display 2. Whendisplaying the fluoroscopic image V16, the controller 9 processes aportion corresponding to the outside of the human body to betransparent. As a result, as depicted in FIG. 18A, an area A1 isprovided around the fluoroscopic image V16. Through the area A1, theuser can view the other side of the display device 1.

In FIG. 18A, the display device 1 displaying the fluoroscopic image V16,which is a fluoroscopic image of the abdomen, is held in front of theabdomen of the patient. By holding the display device 1 in this manner,the user of the display device 1 can treat the fluoroscopic image as ifthe inside of the abdomen of the patient is see through.

The controller 9 that has displayed the fluoroscopic image V16 at S61determines whether the user has touched the switching button V17 (S62).When the controller 9 determines that the user has not touched theswitching button V17, the controller 9 continues to display thefluoroscopic image V16. When the controller 9 determines that the userhas touched the switching button V17, the controller 9 obtains a vectorindicating the current orientation of the display device 1 from thesecond detector 6 and stores the vector in the storage 8 (S63).Subsequently, the controller 9 enters the non-AR mode (S64) and sets thetransmittance of the display 2 to the minimum value. The controller 9displays, on the display 2, a cross-sectional image that has been storedin the storage 8 in association with the same angle as the fluoroscopicimage V16 displayed at S61 (S65). In the example in FIG. 18A, since thefluoroscopic image V16 is captured from the front of the patient, thecontroller 9 displays a coronal image V20 depicted in FIG. 18B at S65.In this manner, the images are switched naturally for the user.

Subsequently, the controller 9 obtains a vector indicating the currentorientation of the display device 1 from the second detector 6 again andcalculates an angle (rotation matrix) between the vector indicating thecurrent orientation and the vector stored at S63 (S66). Then, thecontroller 9 updates the display on the display 2 with a cross-sectionalimage corresponding to the calculated angle (S67). For example, when thecalculated angle indicates that the display device 1 has been turnedupward by 90°, the controller 9 updates the display on the display 2with an axial image V21 depicted in FIG. 18C. When the calculated angleindicates that the display device 1 has been turned rightward by 90°,the controller 9 updates the display on the display 2 with a sagittalimage V22 depicted in FIG. 18D.

After S67, the controller 9 determines again whether the user hastouched the switching button V17 (S68). When the controller 9 determinesthat the user has not touched the switching button V17, the controller 9returns to S66 to continue to display the cross-sectional image in thenon-AR mode. When the controller 9 determines that the user has touchedthe switching button V17, the controller 9 returns to S60 to display afluoroscopic image in the AR1 mode. In subsequent S61, the controller 9may display a fluoroscopic image, not depicted, corresponding to theangle calculated last.

As described above, according to the present embodiment, when thecontroller 9 is in the AR1 mode, the controller 9 displays afluoroscopic image. When the controller 9 is in the non-AR mode, thecontroller 9 displays a cross-sectional image. Moreover, when thecontroller 9 is switched from the AR1 mode to the non-AR mode, thecontroller 9 displays the cross-sectional image captured at the sameplane as the imaging plane of the fluoroscopic image that has beendisplayed. In this manner, the images are switched naturally for theuser. Further, while displaying the cross-sectional image in the non-ARmode, the controller 9 generates a cross-sectional image from volumedata according to the tilt of the display device 1. As a result, thecross-sectional images are switched naturally for the user.

In the present embodiment, when the user inputs to the display 2 usingthe electronic pen 10 while a fluoroscopic image or a cross-sectionalimage is displayed, stroke data generated by this input may be stored inthe storage 8 in association with the image being displayed. This allowsthe user to check the input contents later again.

An eighth embodiment of the present disclosure will now be described. Inaddition to the functions described in the first embodiment, the displaydevice 1 according to the present embodiment has a function ofdisplaying additional information such as a correct answer to an examquestion.

FIGS. 20A and 20B are diagrams depicting how the display device 1according to the present embodiment is used. When the controller 9according to the present embodiment activates an additional-informationdisplaying function, the controller 9 authenticates the user using an IDand password, for example. With this configuration, only a person (anexaminer, for example) who knows the ID and password can activate theadditional-information displaying function.

The storage 8 according to the present embodiment stores, in advance,the structure of an exam paper for which the additional-informationdisplaying function is activated. The structure includes the position ofeach of one or more answer columns. The storage 8 also stores a correctanswer for each answer column in advance. An exam paper 34 depicted inFIG. 20A is an example of the exam paper and includes one or more answercolumns 34 a.

The controller 9 activates the additional-information displayingfunction and enters the AR1 mode or the AR2 mode. Subsequently, thecontroller 9 starts capturing a video using the imager 4. The controller9 causes the imager 4 to continue to capture the video until the exampaper stored in the storage 8 appears in the video output from theimager 4. When the exam paper stored in the storage 8 has appeared, thecontroller 9 determines the position of each of the answer column(s)included in the exam paper in the video on the basis of the position ofthe exam paper in the video. Since the display device 1 does not usuallystay at one place, the controller 9 continuously determines the positionof each of the answer column(s) at a predetermined interval.

Each time the controller 9 has determined the position of each of theanswer column(s), the controller 9 determines whether the determinedposition is included inside the display 2. For the answer columndetermined to be included inside the display 2, the controller 9displays a correct answer at a corresponding position within the display2. The correct answer to be displayed is stored in the storage 8 inassociation with each of the answer column(s). In this manner, the usercan view the correct answer of each of the answer column(s) on thedisplay 2.

The paper for which the present embodiment is applied is not limited toan exam paper. Similarly to the exam paper, the display device 1 iscapable of displaying a correct answer for each of one or more answercolumns provided in a textbook, a reference book, a workbook, or thelike.

Additionally, when the user touches an area inside the display 2 thatcorresponds to one of the answer columns, the controller 9 may displayfurther additional information (for example, explanations) related tothe answer column. In this case, the additional information is stored inadvance in the storage 8 in association with each of the answercolumn(s). The controller 9 may enter the non-AR mode and display theadditional information using the entire display 2. In addition, thecontroller 9 may display a switching button at the same time whendisplaying the additional information. When the user touches theswitching button, the controller 9 may return to display the correctanswer for the answer column.

Although the embodiments of the present disclosure have been describedabove, the present disclosure is not limited to these embodiments andmay be implemented in various other embodiments without departing fromthe scope of the present disclosure.

For example, in each of the embodiments above, description has been madeon the premise that the display device 1 supports three of the non-ARmode, the AR1 mode, and the AR2 mode. However, at least part of thepresent disclosure may be implemented by the display device 1 supportingone of the AR1 mode and the AR2 mode. For example, in the example inFIGS. 4A to 4C, even if the display device 1 does not support the AR2mode, the display mode of the virtual image may be switched between thenon-AR mode in FIG. 4A and the AR1 mode in FIG. 4B. Further, even if thedisplay device 1 does not support the AR1 mode, the display mode of thevirtual image may be switched between the non-AR mode in FIG. 4A and theAR2 mode in FIG. 4C.

In each of the above-described embodiments, the display device 1 has acircular shape. Alternatively, the display device 1 may have anothershape such as a quadrangle.

What is claimed is:
 1. A display device comprising: a display havingtransparency; a first detector which, in operation, detects a positionindicated by an indicator on the display; a second detector which, inoperation, detects position information and an orientation of thedisplay; and a controller which, in operation, displays a virtual imagecorresponding to the position information and the orientation on thedisplay and changes a display mode of the virtual image according to theposition indicated by the indicator.
 2. The display device according toclaim 1, wherein: the controller, in operation, changes a transmittanceof the display.
 3. The display device according to claim 1, wherein: atransmittance of the display is changeable, the controller is operatesin one of a plurality of modes including at least: a first augmentedreality mode in which the controller sets the transmittance of thedisplay to a first value; and a non-augmented reality mode in which thecontroller sets the transmittance of the display to a second valuesmaller than the first value to display the virtual image on thedisplay, and the controller, in operation, further displays a firstimage on the display while operating in the first augmented realitymode, and when the position indicated by the indicator includes aposition on the first image, the controller changes the display mode ofthe virtual image by switching the controller from the first augmentedreality mode to the non-augmented reality mode.
 4. The display deviceaccording to claim 3, wherein: the plurality of modes includes a secondaugmented reality mode in which the controller sets the transmittance ofthe display to a third value smaller than the first value andsuperimposes the virtual image on a video output from an imager todisplay the virtual image superimposed on the video on the display. 5.The display device according to claim 1, wherein: a transmittance of thedisplay is changeable, and the controller is which, in operation,further display a second image on the display and control thetransmittance of the display based on the position indicated by theindicator on the second image.
 6. The display device according to claim1, wherein: the controller is which, in operation, generate stroke databased on the position indicated by the indicator and display the strokedata on the display.
 7. The display device according to claim 1,wherein: the controller is which, in operation, change an enlargementratio of the virtual image.
 8. A method for controlling a display deviceincluding a display having transparency, the method comprising:detecting a position indicated by an indicator on the display; detectingposition information and an orientation of the display; displaying avirtual image corresponding to the position information and theorientation on the display; and controlling changing of a display modeof the virtual image according to the position indicated by theindicator.
 9. The method for controlling the display device according toclaim 8, wherein: the controlling includes changing a transmittance ofthe display.
 10. The method for controlling the display device accordingto claim 8, wherein: a transmittance of the display is changeable, thedisplay device operates in one of a plurality of modes including atleast: a first augmented reality mode in which the display device setsthe transmittance of the display to a first value; and a non-augmentedreality mode in which the display device sets the transmittance of thedisplay to a second value smaller than the first value to display thevirtual image on the display, and the displaying further includesdisplaying a first image on the display while the display deviceoperates in the first augmented reality mode, and the controllingfurther includes changing the display mode of the virtual image byswitching the display device from the first augmented reality mode tothe non-augmented reality mode when the position indicated by theindicator includes a position on the first image.
 11. The method forcontrolling the display device according to claim 10, wherein: theplurality of modes includes a second augmented reality mode in which thedisplay device sets the transmittance of the display to a third valuesmaller than the first value and superimposes the virtual image on avideo output from an imager to display the virtual image superimposed onthe video on the display.
 12. The method for controlling the displaydevice according to claim 8, wherein: a transmittance of the display ischangeable, the displaying further includes displaying a second image onthe display, and the controlling includes controlling the transmittanceof the display based on the position indicated by the indicator on thesecond image.
 13. The method for controlling the display deviceaccording to claim 8, wherein: the controlling includes generatingstroke data based on the position indicated by the indicator anddisplaying the stroke data on the display.
 14. The method forcontrolling the display device according to claim 8, wherein: thecontrolling includes changing an enlargement ratio of the virtual image.